Apparatus and method for data embedding in light communication and the light communication system and method thereof

ABSTRACT

In a light communication system, a data embedding unit arranged between a transmitter-side communication data processing unit and a light emitting device driver embeds a communication processed data at a spatial domain of an original image according to a modulation scheme, and gets multiple RGB values for a communication data embedded image. A receiving apparatus detects a transmitter-side communication data embedded image, generates a receiver-side communication data embedded image, compensate a deformation of the receiver-side communication data embedded image, outputs a warped communication data embedded image, and extracts a communication processed data from the warped communication data embedded image.

TECHNICAL FIELD

The present disclosure relates generally to an apparatus and method fordata embedding in light communication and the light communication systemand method thereof.

BACKGROUND

Light communication is a wireless communication technology that useshuman visible light with frequency among 400 THz to 800 THz. RFbandwidth is a scarce resource. Therefore, visible light communicationmay provide an alternate technology to meet the strong demands ofwireless communications. For example, the visible light emitted from oneor more light emitting diodes (LEDs) is widely used in homes andoffices, thus it makes the visible light emitted from the one or moreLEDs ideal for ubiquitous data transmitters. FIG. 1A and FIG. 1B showschematic views of two exemplary types of white-light LEDs,respectively. FIG. 1A shows an exemplary white-light LED using a blueLED 110 and a yellow phosphor 115. The yellow phosphor 115 limits themodulation bandwidth no more than about 10 Mbps. FIG. 1B shows anotherexemplary type of a white-light LED using a RGB triplet that may includea blue LED 121, a red LED 122 and a green LED 123, and these three LEDsemit blue light 131, red light 132 and green light 133, respectively.Compared with the white-light LED shown in FIG. 1A, the RGB triplet hasa potentially higher bandwidth and the mixing of the triplet maygenerate any desired colors by any combination of a blue light, a redlight and a green light. Therefore, a RGB triplet LED array may be usedas a high-bit-rate data transmitter and a lighting device at the sametime.

FIG. 2A shows a schematic view of a general transmitter 200 using RGBtriplet LEDs with a single input data stream 212. In the transmitter200, the single input data stream 212 is received by an error correctionencoder 214, and the output of a modulator 216 is applied to a tripletLED driver 218 directly. Three output powers P_(R), P_(G) and P_(B) ofthree primary colors (i.e. red, green and blue) are generated by thetriplet LED driver 218. FIG. 2B shows an example of the output of themodulator with an on-off keying modulation scheme. The bit streamoutputted by the modulator 216 is distributed to the triplet LEDssequentially, wherein R, G and B represent three bit streams of threeprimary colors, respectively. FIG. 2C shows an example of the outputpowers of the triplet LED driver.

FIG. 3A shows a schematic view of a wavelength division multiplexing(WDM) transmitter 300 using RGB triplet LEDs with multiple input data.In the transmitter 300, each of multiple input data such as input data0,input data1 and input data2, has individual error correction encoder andmodulator. The outputs of three modulators are coupled to three LEDdrivers directly. FIG. 3B shows an example of the output powers of thethree LED drivers.

FIG. 4 shows a schematic view of a technique for visible lightcommunication (VLC). As shown in the FIG. 4, a VLC apparatus maycomprise a transmitting side device 410 and a reception side device 420.The transmitting side device 410 for transmitting a plurality ofcommunication data (such as data 1, data 2, data 3, etc.) includes anilluminator 412 for generation of the illumination light, acommunication amount adjuster 413 and a modulator 411. The communicationamount adjuster 413 receives each of a plurality of communication datasignals, and generates a dummy data for a corresponding one of theplurality of individual light sources so that data to be transmittedthrough each of light sources has an equal communication amount. Themodulator 411 modulates each of the received communication data signalsand the received dummy data into a driving signal for said each of theplurality of light sources.

FIG. 5 shows a schematic view illustrating a data transmitting apparatususing visible light communication. As shown in the FIG. 5, the datatransmitting apparatus 510 may comprise a code generator 512 thatconverts transmitting data to a two-dimensional data code 520 havingdifferent colors and patterns according to times T₀, T₁, . . . ,T_(N-1), T_(N), a modulation unit 514 that generate a modulation signalby modulation the two-dimensional data code, a plurality of LEDs 518that are arranged in a two-dimensional form and emit light, and a lightsource driver 516 that controls the turn-on of the LEDs 518 according tothe modulation signal.

From the aforementioned technologies, it may be seen that if the outputof the modulator(s) or modulation unit(s) is applied to the LEDdriver(s) directly, the respective energy distribution of the threeprimary colors may be different from one another. Therefore, there maybe an undesirable color tone in the situation. Also, the time varyingluminance may be sensible by the human eyes when the transmitter servesas a lighting device at the same time. Light communication is still apotential solution to the global wireless spectrum shortage. Varioussolutions for visible light communication techniques have beensuggested. There are challenges in these solutions for using anapparatus to perform image or video display and light communicationsimultaneously.

SUMMARY

The exemplary embodiments of the disclosure may provide an apparatus andmethod for data embedding in light communication and the lightcommunication system and method thereof.

One exemplary embodiment relates to an apparatus for data embedding inlight communication, adapted to a transmitter. The apparatus for dataembedding in light communication may comprise a data embedding unitarranged between a transmitter-side communication data processing unitand a light emitting device driver in the transmitter. The dataembedding unit embeds a communication processed data at a spatial domainof an original image according to a modulation scheme, and gets multipleRGB values for a communication data embedded image.

Another exemplary embodiment relates to a method for data embedding inlight communication, adapted to a transmitter. The method for dataembedding in light communication may comprise: arranging a dataembedding unit between a transmitter-side communication data processingunit and a light emitting device driver in the transmitter; embedding,by using the data embedding unit, a communication processed data at aspatial domain according to a modulation scheme to generate acommunication modulated image; and performing an inverse transform and adomain transform that transforms the communication modulated image froma frequency domain into the spatial domain, and getting multiple RGBvalues for a communication data embedded image.

Yet another exemplary embodiment relates to a receiving apparatus fordata embedding in light communication. The receiving apparatus maycomprise an image sensing device, an image warping unit, and a dataextraction unit. The image sensing unit detects a transmitter-sidecommunication data embedded image transmitted by a light emittingdevice, and generates a receiver-side communication data embedded image.The image warping unit compensates a deformation of the receiver-sidecommunication data embedded image by performing an image warpingprocessing on the transmitter-side communication data embedded image,and outputs a warped communication data embedded image. The dataextraction unit extracts a communication processed data from the warpedcommunication data embedded image.

Yet another exemplary embodiment relates to a receiving method for dataembedding in light communication. The receiving method may comprise:configuring an image warping unit and a data extraction unit between animage sensing device and a receiver-side communication data processingunit; detecting a transmitter-side communication data embedded imagetransmitted by a light emitting device and generating a receiver-sidecommunication data embedded image, by using the image sensing device;performing an image warping processing on the transmitter-sidecommunication data embedded image and outputting a warped communicationdata embedded image, by using the image warping unit; and extracting acommunication processed data from the warped communication data embeddedimage, by using the data extraction unit.

Yet another exemplary embodiment relates to a light communicationsystem. The light communication system may comprise a transmitter and areceiving apparatus. The transmitter embeds a communication processeddata at one or more frequency coefficients of a frequency domaintransform of an original image, performs an inverse transform and adomain transform on the communication modulated image, and forms acommunication data embedded image by embedding multiple RGB values inthe original image. The receiving apparatus detects the communicationdata embedded image transmitted by a light emitting device and generatesa receiver-side communication data embedded image, and performs an imagewarping processing to recover the communication data embedded image bythe receiver-side communication data embedded image, and extracts thecommunication processed data from a warped communication data embeddedimage.

Yet another exemplary embodiment relates to a light communicationmethod. The light communication method may comprise: in a transmitter,embedding a communication processed data at one or more frequencycoefficients of a frequency domain transform of an original image,performing an inverse transform and a domain transform on thecommunication modulated image, and forming a communication data embeddedimage by embedding multiple RGB values in the original image; and in areceiving apparatus, detecting the communication data embedded imagetransmitted by a light emitting device and generating a receiver-sidecommunication data embedded image, and performing an image warpingprocessing to recover the communication data embedded image by thereceiver-side communication data embedded image, and extracting thecommunication processed data from a warped communication data embeddedimage.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1A and FIG. 1B show schematic views of two exemplary types ofwhite-light LEDs, respectively.

FIG. 2A shows a schematic view of a general transmitter using RGBtriplet LEDs with a single input data stream.

FIG. 2B shows an example of the output of the modulator with an on-offkeying modulation scheme.

FIG. 2C shows an example of the output powers of the triplet LED driver.

FIG. 3A shows a schematic view of a WDM transmitter using RGB tripletLEDs with multiple input data streams.

FIG. 3B shows an example of the output powers of the three LED drivers.

FIG. 4 shows a schematic view of a technique for visible lightcommunication.

FIG. 5 shows a schematic view illustrating a data transmitting apparatususing visible light communication.

FIG. 6 shows a schematic view of a light communication system, accordingto an exemplary embodiment.

FIG. 7 shows an apparatus for data embedding in light communication,adapted to a transmitter, according to an exemplary embodiment.

FIG. 8 shows a block diagram of the apparatus for data embedding inlight communication including a preprocessor, according to one exemplaryembodiment.

FIG. 9 shows the operation of the data embedding unit, by taking anoriginal input image as an example, according to an exemplaryembodiment.

FIG. 10 shows a first exemplar for the communication processed dataembedding with a BPSK modulation scheme, according to an exemplaryembodiment.

FIG. 11 shows a second exemplar for the communication processed dataembedding with a BPSK modulation scheme, according to an exemplaryembodiment.

FIG. 12 shows a third exemplar for the communication processed dataembedding with a QPSK modulation scheme, according to an exemplaryembodiment.

FIG. 13 shows a fourth exemplar for the communication processed dataembedding by a mixed BPSK and QPSK modulation scheme, according to anexemplary embodiment.

FIG. 14 shows a fifth exemplar for the communication processed dataembedding with a spread spectrum sequence, according to an exemplaryembodiment.

FIG. 15 shows a first exemplar that embeds the communication modulatedimage in the spatial YUV domain with the original image, according to anexemplary embodiment.

FIG. 16 shows a second exemplar that embeds the communication modulatedimage in the spatial RGB domain with the original image, according to anexemplary embodiment.

FIG. 17 shows a method for data embedding in light communication,adapted to a transmitter, according to an exemplary embodiment.

FIG. 18 shows a receiving apparatus for data embedding in lightcommunication, according to an exemplary embodiment.

FIG. 19 shows a barycentric coordinate conversion of a point in atriangle in the image warping processing, according to an exemplaryembodiment.

FIG. 20 shows the operation of the image warping processing, accordingto an exemplary embodiment.

FIG. 21 shows the operation of the data extraction unit, according to anexemplary embodiment.

FIG. 22 shows a receiving method for data embedding in lightcommunication, according to an exemplary embodiment.

DETAILED DESCRIPTION OF DISCLOSED EMBODIMENTS

Below, exemplary embodiments will be described in detail with referenceto accompanying drawings so as to be easily realized by a person havingordinary knowledge in the art. The inventive concept may be embodied invarious forms without being limited to the exemplary embodiments setforth herein. Descriptions of well-known parts are omitted for clarity,and like reference numerals refer to like elements throughout.

The exemplary embodiments in the disclosure provide a technique for dataembedding in light communication to perform image or video display andlight communication simultaneously. The technique employs a dataembedding technique for the communication processed data before applyingthe communication processed data to the light emitting device driver orthe image/video display driver. Instead of directly applying thecommunication processed data to the light emitting device driver such asLED driver, the disclosed exemplary embodiments embed the communicationprocessed data on the eye insensitive part of the frequency coefficientsin the spatial domain of the image or the video when light communicationis performed.

As seen in FIG. 6, one exemplary embodiment of a light communicationsystem 600 may comprise a transmitting side device 610 and a receivingside device 620. The transmitting side device 610 may comprise atransmitter-side communication data processing unit 612, a dataembedding unit 614, a light emitting device driver 616 and a lightemitting device 618. The transmitter-side communication data processingunit 612 may perform a signal processing of communication input data 612a. The data embedding unit 614 may embed the communication processeddata 614 a at one or more frequency coefficients of a frequency domaintransform of an original image 614 b according to a modulation codingscheme. Before embedding the communication processed data 614 a, anexternal indicator 614 c may be used to indicate whether or not apre-processing procedure is performed. The external indicator 614 c maybe generated by a software module or a hardware element. The lightemitting device driver 616 may drive the light emitting device 618according to the output data processed by the data embedding unit 614.

The receiving side device 620 may comprise an image sensing device 622,an image warping unit 624, a data extraction unit 626 and areceiver-side communication data processing unit 628. The image sensingunit may detect a communication data embedded image 618 b transmittedfrom the light emitting device 618 via a channel. The image warping unit624 may perform an image warping functions to compensate the deformationof a sensed image from the image sensing unit. The data extraction unit626 may extract the communication processed data 614 a from an output ofthe image warping unit 624. The receiver-side communication dataprocessing unit 628 performs the common communication data processingsuch as demodulation and error correction, to recover the communicationinput data 612 a from the communication processed data extracted by thedata extraction unit 626.

As mentioned in FIG. 6, the transmitter-side communication dataprocessing unit may perform the common communication signal processingin the transmitting side device, such as error correction encoding andmodulation etc, to generate the communication processed data. Thecommunication processed data may be, but not limited to, a bit stream ofa stream of signals, such as I/Q pairs. FIG. 7 shows an apparatus fordata embedding in light communication, adapted to a transmitter 700,according to an exemplary embodiment. Referring to FIG. 7, the apparatusfor data embedding in light communication may comprise the dataembedding unit 614 arranged between a transmitter-side communicationdata processing unit 612 and a light emitting device driver 616 in thetransmitter 700. The data embedding unit 614 may embed the communicationprocessed data 614 a at one or more frequency coefficients of afrequency domain transform such as a discrete cosine transform (DCT) ofthe original image 614 b (step 720) according to a modulation scheme.The original image 614 b without embedded communication data may be astill image or a frame in a video sequence. After embed the one or morefrequency coefficients of a frequency domain transform, the dataembedding unit performs a domain transform such as an inverse DCT (IDCT)that transforms the communication modulated image from the frequencydomain into a spatial domain transform (step 730), and gets multiple RGBvalues 732 for a communication data embedded image 618 b such as bytransferring the luminance and chrominance signals to RGB signals.

In other words, the data embedding unit 614 may generate a communicationmodulated image 722, and gets multiple RGB values for a communicationdata embedded image. The data embedding unit may embed the communicationprocessed data by inserting one or more frames modulated by one or moreimage intensities inter a plurality of original frames over the originalimage. The one or more frames inserted are modulated by the one or moreintensities of one or more partial images in different regions over theoriginal image.

The data embedding unit may generate a communication modulated image byembedding the communication processed data at one or more frequencycoefficients of a frequency domain transform of the original image. Thedata embedding unit may generate the communication data embedded imageby embedding the communication modulated image in the original image ina frequency domain, performing the frequency domain transform, andtransferring a plurality of luminance and chrominance data to themultiple RGB values. Or the data embedding unit may perform a domaintransform from a frequency domain to a spatial RGB domain for thecommunication modulated image, and transferring a plurality of resultedluminance and chrominance data to the multiple RGB values, and generatesthe communication data embedded image by embedding the multiple RGBvalues in the original image. Or the data embedding unit may perform adomain transform from a frequency domain to the spatial domain for thecommunication modulated image, and generate the communication dataembedded image by embedding a transformed result with the original imageand transferring a plurality of resulted luminance and chrominance dataof a data embedded image to the multiple RGB values.

Before embedding the communication processed data, the apparatus fordata embedding in light communication may perform a pre-processingprocedure according to the external indicator 614 c. For example, thepre-processing procedure may include transferring the pixel arrays ofthe original image 614 b from a first domain such as a RGB domain to asecond domain such as a YUV (or YCbCr) domain and performing the DCT onthe pixel arrays in the YUV domain. The external indicator 614 cindicates whether or not the pre-processing procedure is performed. Theexternal indicator may be generated by a software module or a hardwareelement. The data embedding unit may embed the communication processeddata at one or more predefined frequency coefficients of such as thediscrete cosine transform (DCT) of the original image. The lightemitting device driver drives the light emitting device according to thesignal processed by the data embedding unit. The output of the lightemitting device 618 in the transmitter is a communication data embeddedimage to be transmitted for light communication.

Accordingly, as shown in an exemplary embodiment of FIG. 8, theapparatus for data embedding in light communication shown in FIG. 7 mayfurther include a preprocessor 810 configured to perform apre-processing procedure 812 before embedding the communicationprocessed data, according to the external indicator 614 c. After thepre-processing procedure, the data embedding unit 614 embeds thecommunication processed data 614 at the one or more the frequencycoefficients of a spatial domain (i.e. perform the steps 720 and 730),and gets the multiple RGB values 732 of the communication data embeddedimage 618 b.

FIG. 9 shows the operation of the apparatus for data embedding in lightcommunication, adapted to a transmitter, by taking an original image asan example, according to an exemplary embodiment. In this embodiment,the original input image is composed of 8×8 RGB pixel arrays 912. Whenthe external indicator 614 c indicates the pre-processing procedureshould be performed, the preprocessor 810 performs a pre-processingprocedure before embedding the communication processed data, such astransferring the pixel arrays 912 of an original image from a RGB domainto another domain such as a YUV (or YCbCr) domain (i.e. step 910 ofRGB2YUV) to generate pixel arrays in the YUV (or YCbCr) domain 922 andperforming a DCT (i.e. step 920) on the pixel arrays 922 in the YUVdomain to generate pixel arrays 932 of a communication modulated imageafter the DCT. The external indicator 614 c may be generated by using asoftware programming register or a hardware element.

The data embedding unit 614 may embed the communication processed data614 a at the one or more frequency coefficients of any combination ofthe Y, U and V parts of the DCT blocks (i.e. step 930 of embeddingfrequency coefficients of any combination of the Y, U and V parts)according to a modulation coding scheme, thereby producing, for example,three pixel arrays 932 that have been embedded the communicationprocessed data 614 a on the frequency coefficients of a spatial domain.The data embedding unit 614 may remove the coefficients of the originalinput image and add at least one value according to the communicationprocessed data 614 a and the modulation type. The data embedding unit614 may perform an IDCT (step 940) to recover the pixel arrays 922 and aYUV (YCbCr) to RGB transform (i.e. step 950) sequentially to obtainmultiple RGB values 952 of a communication data embedded image. In thisexample, the communication data embedded image 932 is formed byembedding the obtained RGB values 952 in the 8×8 RGB pixel arrays 912,and is to be transmitted by the transmitter. The RGB values 952 will beapplied to a light emitting device driver that drives the light emittingdevice to output a communication data embedded image.

By properly choosing the one or more frequency coefficients forcommunication processed data embedding, for example but not limits tothe middle-low frequency coefficients, the distortion after embeddingthe communication processed data may not be sensible by human eyes. Thefollowing examples illustrate some schema for the communicationprocessed data embedding. FIG. 10 shows a first exemplar for thecommunication processed data embedding with a BPSK modulation scheme,according to an exemplary embodiment. In the exemplar of FIG. 10, animage is for example, but not limited to, composed of a plurality of 8×8pixel arrays, and each of a plurality of DCT blocks is of a size 8×8.Here the communication processed data is a binary bit stream representedby {b₀₀, b₀₁, . . . , b₁₃}, where b_(ij) denotes the jth-bit embedded ina i-th frame. Let Y₀, Y₁ be two Y components of a DCT result at timeinstance 0, 1 respectively. Let {f_(i0), f_(i1), . . . , f_(i(N-1))} beN frequency coefficients for embedding the communication processed data,where N is the number of the frequency coefficients in each DCT block.For the BPSK modulation scheme, the values of f_(i0), f_(i1), . . . ,and f_(i(N-1)) are set to be f_(i0)=f_(i1)= . . . =f_(i(N-1))=+m orf_(i0)=f_(i0)= . . . =f_(i(N-1))=−m according to the input bit in thecommunication processed data. In the exemplar of FIG. 10, threefrequency coefficients marked by three slash squares, respectively, areused for embedding the communication processed data with the BPSKmodulation. In this case, we have N=3 and m=4. The 12 low and middlecoefficients of the 4 DCT blocks on the 1^(St) row are all set to thevalue −4 or 4 according to b₀₀, the 12 low and middle frequencycoefficients of the 4 DCT blocks on the 2^(nd) row are all set to thevalue of b₀₁, and so on.

FIG. 11 shows a second exemplar for the communication processed dataembedding by a BPSK modulation scheme in an interleaved fashion,according to an exemplary embodiment. In the exemplar of FIG. 11, thecommunication processed data {b₀₀, b₀₁, . . . , b₁₇} is embedded andinterleaved in the low and middle coefficients of DCT blocks with a BPSKmodulation scheme. When the input bit value of the communicationprocessed data is 1, the low and middle coefficients (represented by theslash squares) of the associated DCT blocks are set to the value 4;otherwise, the coefficients are set to −4. In this exemplar, the 6 lowand middle coefficients of two DCT blocks on the 1^(st) row are setaccording to the value of b₀₀, and the 6 low and middle coefficients ofthe other two DCT blocks on the 1^(st) row are set according to thevalue of b₀₁, and so on.

FIG. 12 shows a third exemplar for the communication processed dataembedding with a QPSK pattern, according to an exemplary embodiment. Inthe exemplar, each of a plurality of DCT blocks is of size 8×8 and eachsmall square in represents a coefficient in a DCT block. Thecommunication processed data is composed of QPSK symbols. Let {f_(i0)_(—) ₀₀, f_(i1) _(—) ₀₀, . . . , f_(i(N-1)) _(—) ₀₀}, {f_(i0) _(—) ₀₁,f_(i1) _(—) ₀₁, . . . , f_(i(N-1)) _(—) ₀₁}, {f_(i0) _(—) ₁₀, f_(i1)_(—) ₁₀, . . . , f_(i(N-1)) _(—) ₁₀} and {f_(i0) _(—) ₁₁, f_(i1) _(—)₁₁, . . . , f_(i(N-1)) _(—) ₁₁} be the N frequency coefficients in eachDCT block for 4 QPSK symbols 00, 01, 10, 11 respectively, where N is thenumber of the frequency coefficients in each DCT block. In the exampleof FIG. 12, we have N=3. When the QPSK symbols in the communicationprocessed data are 00, 01, 10, 11, the low and middle coefficients(represented by slash squares) of a corresponding DCT block are replacedby {−4, −4, −4}, {−4, 4, 4}, {4, −4, −4} and {4, 4, 4} respectively.

FIG. 13 shows a fourth exemplar for the communication processed dataembedding by a mixed BPSK and QPSK modulation scheme, according to anexemplary embodiment. In the exemplar, the pixel arrays and the DCTblocks are both of the size 8×8, and two modulation schemes are mixed ina block. Since an image may suffer from noise and geometricaldeformation when receiving from an image sensing device, thecoefficients of the middle frequencies are more reliable than those ofthe high frequencies. Thus the communication processed data may beembedded in the high frequency coefficients with a strong modulationscheme, and in the middle frequency coefficients with a weakermodulation scheme. In this exemplar, the communication processed data isembedded in the middle frequency coefficients with the QPSK modulation,and in the high frequency coefficients with the BPSK modulation.

FIG. 14 shows a fifth exemplar for the communication processed dataembedding with a spread spectrum sequence, according to an exemplaryembodiment. In the exemplar, the pixel arrays and the DCT blocks areboth of the size 8×8. The communication processed data is represented bya binary bit stream. Let {f_(i0) _(—) ₀, f_(i1) _(—) ₀, . . . ,f_(i(N-1)) _(—) ₀} and {f_(i0) _(—) ₁, f_(i1) _(—) ₁, . . . , f_(i(N-1))_(—) ₁} be the N frequency coefficients in each DCT block for embeddingbit 0 and bit 1 respectively, where N is the number of the frequencycoefficients in each DCT block, and the values of these f_(i) _(—) _(j)may be different. In the exemplar of FIG. 14, we have N=28, and acommunication processed data with bit value 0 being embedded in thefrequency coefficients with a spread spectrum pattern (sequence) 1410,and that with bit value 1 being embedded in the coefficients with aspread spectrum pattern (sequence) 1420. Different combinations of thefrequency coefficients may be used to represent the spread spectrumcoding. The receiver may extract the coefficients in the frequencydomain as input signals, and calculate the correlation with the twospread spectrum sequences to recover the communication processed data.For example, when the correlation between an input signals and thesequence of value 0 is greater than the the correlation between an inputsignals and the sequence of value 1, the spread spectrum code is 0;otherwise, the spread spectrum code is 1.

There are different schemes in the transmitter for the communicationprocessed data embedding with the original image. Two exemplars areshown in FIG. 15 and FIG. 16, respectively. FIG. 15 shows a firstexemplar that embeds the communication modulated image in the spatialYUV domain with the original image, according to an exemplaryembodiment. In the exemplar of FIG. 15, before embedding thecommunication processed data, the preprocessor such as a RGB2YUV 910 maytransfer the pixel arrays of the original image from a first spatialdomain such as a RGB domain to a second spatial domain such as a YUV (orYCbCr) domain. After the preprocessing, the data embedding unit 614 mayembed the communication processed data at the coefficients in thefrequency domain, such as embed frequency coefficients of anycombination of the Y,U, and V parts as labeled by 930, according to amodulation scheme and generates the communication modulated image 1510with the embedded frequency coefficients, and performs an IDCTtransform. The data embedding unit generates a communication dataembedded image 1520 in the spatial YUV domain by embedding thecommunication modulated image 1510 with the original image in thespatial YUV domain. Finally, the data embedding unit transforms thecommunication data embedded image 1520 from the spatial YUV (YCbCr)domain to the spatial RGB domain, to get the RGB values of thecommunication data embedded image in the spatial RGB domain for lightcommunication.

FIG. 16 shows a second exemplar that embeds the communication modulatedimage in the spatial RGB domain with the original image, according to anexemplary embodiment. In the exemplar of FIG. 16, no preprocessingprocess is performed. The data embedding unit 614 embeds thecommunication processed data at the coefficients in the frequency domainaccording to a modulation scheme to generate the communication modulatedimage 1510, performs an IDCT transform of the communication modulatedimage 1510, and transforms the communication modulated image 1510 fromthe YUV (YCbCr) domain to the RGB domain to get the RGB values of acommunication modulated image 1620 in the spatial RGB domainsequentially. Finally, the data embedding unit 614 embeds the RGB valuesof the communication modulated image in the RGB values of the originalimage in the spatial RGB domain to generate a communication dataembedded image 1620 in the spatial RGB domain for light communication.

As has been described above, a method for data embedding in lightcommunication, adapted to a transmitter, may be illustrated in FIG. 17,according to an exemplary embodiment. In the FIG. 17, the method mayarrange a data embedding unit between a transmitter-side communicationdata processing unit and a light emitting device driver in thetransmitter (step 1710), and embeds, by using the data embedding unit, acommunication processed data at one or more frequency coefficients of afrequency domain of an original image according to a modulation schemeto generate a communication modulated image (step 1720), performs aninverse transform and a domain transform that transforms thecommunication modulated image from the frequency domain into a spatialdomain, and gets multiple RGB values of a communication data embeddedimage (step 1730), wherein the communication data embedded image isformed by embedding the multiple RGB values with the original image.

As mentioned earlier, when a preprocessing process is performedaccording an external indicator, the original image may be transformedfrom a RGB spatial domain to a spatial YUB domain. After thepreprocessing process has completed and the communication modulatedimage is generated, the method may generate the communication dataembedded image by embedding the communication modulated image with theoriginal image in the spatial YUV domain, transform the communicationdata embedded image from the spatial YUV domain to the spatial RGBdomain, to get the multiple RGB values of the communication dataembedded image for light communication. When no preprocessing process isperformed, the method may transform the communication modulated imagefrom the YUV (YCbCr) domain to the RGB domain after having performed anIDCT transform, to get the RGB values of the communication modulatedimage in the spatial RGB domain sequentially, embed the RGB values ofthe communication modulated image in the RGB values of the originalimage in the spatial RGB domain to generate a communication dataembedded image in the spatial RGB domain for light communication.

As has been described above, the method for data embedding in lightcommunication may embed the communication processed data at the one ormore frequency coefficients of any combination of the Y, U and V partsof the DCT blocks according to a modulation coding scheme. Thecommunication processed data embedding with different modulation schemessuch as, but not limited to, the examples shown in FIG. 10 to FIG. 14may be applied to the method.

When a communication data embedded image is transmitted by the lightemitting device 618 via a channel for light communication, one exemplaryembodiment of a receiving apparatus for data embedding in lightcommunication may refer to FIG. 18. In FIG. 18, the receiving apparatusfor data embedding in light communication 1800 may comprise an imagesensing device 1822, an image warping unit 1824, and a data extractionunit 1826. The image sensing unit 1822 may detect a transmitter-sidecommunication data embedded image 1822 a transmitted by the lightemitting device 618 via a channel, and generate a receiver-sidecommunication data embedded image 1822 b. Because the size and theshaping of the receiver-side communication data embedded image 1822 bmay be different from the transmitter-side communication data embeddedimage 1822 a, the image warping unit 1824 is configured to compensatethe deformation of the receiver-side communication data embedded image1822 b by performing an image warping processing on the transmitter-sidecommunication data embedded image 1822 a, and outputs a warpedcommunication data embedded image 1824 b. The data extraction unit 1826is configured to extract the communication processed data 614 a from thewarped communication data embedded image 1824 b. The receiving apparatusmay further perform a communication data processing 1828 on thecommunication processed data 614 a to obtain the recovered data.

The image warping processing performed by the image warping unit 1824may include a coordinate conversion on a plurality of pixel data in thearea of the transmitter-side communication data embedded image 1822 a,and a coordinate such as a barycentric coordinate may be used in theimage warping process. FIG. 19 shows the barycentric coordinateconversion of a point in a triangle in the image warping processing,according to an exemplary embodiment. Let abc is a triangle defined byits vertices a, b and c. A point p inside the triangle abc may berepresented as αa+βb+γc, where α, β and γ may be calculated by the arearatio of each triangle, and must meet the constraint α+β+γ=1. In otherwords, the point p and the constraint may be expressed by the followingformulas.

p=αa+βb+γc, and α=A _(a) /A,β=A _(b) /A,γ=A _(c) /A,α+β+γ=1,

Where A_(a) is the area of the triangle bcp, A_(b) is the area of thetriangle cap, and A_(c) is the area of the triangle abp, and A is thearea of the triangle abc.

Based on the barycentric coordinate, FIG. 20 shows the operation of theimage warping unit, according to an exemplary embodiment. Referring toFIG. 20, the image warping unit may divide the transmitter-sidecommunication data embedded image 1822 a into several triangles, with atriangle a′ b′ c′ in the transmitter-side communication data embeddedimage 1822 a corresponding to a triangle abc in the receiver-sidecommunication data embedded image 1822 b, and calculate a correspondingbarycentric coordinate (α,β,γ)_(p′) for each point p′ in each trianglea′b′c′, and perform a step 2020 of calculating a corresponding pointp=αa+βb+γc in the receiver-side communication data embedded image 1822 bafter deformation, for each point p′ in each triangle a′b′c′ of thetransmitter-side communication data embedded image 1822 a. And for eachpoint p′ in each triangle a′b′c′ of the transmitter-side communicationdata embedded image 1822 a, the image warping unit may further perform astep 2030 of a pixel value interpolation by the received pixels around pto recover the pixel value of p′. After performing the barycentriccoordinate conversion on the points in the area of the transmitter-sidecommunication data embedded image 1822 a, the pixel values of thetransmitter-side communication data embedded image 1822 a can berecovered by the pixel values of the receiver-side communication dataembedded image 1822 b in the image warping processing.

The operation performed by the data extraction unit 1826 may be shown inFIG. 21, according to an exemplary embodiment. Referring to FIG. 21, thedata extraction unit 1826 may transfer a plurality of data over thewarped communication data embedded image 1824 b from a first spatialrepresentation such as a RGB representation to a second spatialrepresentation such as a YUV (or YCbCr) representation (step 2110),divide the warped communication data embedded image in the secondspatial representation such as the YUV representation into a pluralityof blocks in a frequency domain, and perform a frequency domaintransform (such as DCT or wavelet) on each of the plurality of blocks(step 2120), extract the bits that are embedded in one or more frequencycoefficients (step 2130) to decide the communication processed data 614a by performing a majority voting, demapping or pattern matching on theextracted bits of the one or more frequency coefficients. The receivingapparatus for light communication may further include a receiver-sidecommunication data processing unit that may perform the commoncommunication data processing in the receiving apparatus, such asdemodulation and error correction, to recover the communication datafrom the extracted communication processed data, and outputs therecovered data.

Accordingly, a receiving method for data embedding in lightcommunication may be illustrated in FIG. 22, according to an exemplaryembodiment. In the FIG. 22, the receiving method for data embedding inlight communication may configure an image warping unit and a dataextraction unit between an image sensing device and a receiver-sidecommunication data processing unit (step 2210); detect atransmitter-side communication data embedded image transmitted by thelight emitting device and generate a receiver-side communication dataembedded image, by using the image sensing unit (step 2220); perform animage warping processing on the transmitter-side communication dataembedded image and output a warped communication data embedded image, byusing the image warping unit; and extract the communication processeddata from the warped communication data embedded image, by using thedata extraction unit (step 2230).

For an exemplar of the communication processed data embedded with aspread spectrum sequence, the receiving method for data embedding inlight communication may perform an image warping on a receiver-sidecommunication data embedded image and transfer the data points over awarped communication data embedded image from a RGB representation to aYUV representation, perform a DCT on each such as 8×8 block and extractone or more frequency coefficients of each block, and perform a sequencematching of each block and do a majority voting of a group of blocks.The detected communication processed data may be further applied to areceiver-side communication data processing unit.

As has been described above, one exemplary embodiment of a lightcommunication system may comprise a transmitter and a receivingapparatus. The transmitter may be configured to embed a communicationprocessed data at one or more frequency coefficients of a frequencydomain transform of an original image, to generate a communicationmodulated image, perform an inverse transform and a domain transform onthe communication modulated image, and form a communication dataembedded image by embedding multiple RGB values in the original image.The receiving apparatus may be configured to detect the communicationdata embedded image transmitted by a light emitting device and generatea receiver-side communication data embedded image, and perform an imagewarping processing to recover the transmitter-side communication dataembedded image by the receiver-side communication data embedded image,and extract the communication processed data from a warped communicationdata embedded image.

The transmitter in the light communication system may further include alight emitting device driver to drive the light emitting device totransmit the communication data embedded image. The transmitter may alsoinclude a preprocessor that performs a pre-processing procedure beforeembedding the communication processed data according to an externalindicator. If the pre-processing procedure is not required, thecommunication data embedded image is formed by embedding thecommunication modulated image in a spatial RGB domain with the originalimage in the spatial RGB domain.

Accordingly, according to an exemplary embodiment, in a transmitter, alight communication method may embed a communication processed data atone or more frequency coefficients of a frequency domain transform of anoriginal image, to generate a communication modulated image, perform aninverse transform and a domain transform on the communication modulatedimage, and form a communication data embedded image by embeddingmultiple RGB values in the original image. In a e receiving apparatus,the light communication method may detect the communication dataembedded image transmitted by a light emitting device and generate areceiver-side communication data embedded image, and perform an imagewarping processing to recover the communication data embedded image bythe receiver-side communication data embedded image, and extract thecommunication processed data from a warped communication data embeddedimage.

The details of performing a pre-processing procedure before embeddingthe communication processed data, forming the communication dataembedded image, performing the image warping processing, and extractingthe communication processed data have been described in the earlierexemplary embodiments, and are omitted here.

In summary of the disclosure, the disclosed exemplary embodimentsprovide an apparatus and method for data embedding in lightcommunication, and the light communication system and method thereof.The technique employs a data embedding technique for the communicationprocessed data before applying the communication processed data to thelight emitting device driver or the image/video display driver. Insteadof directly applying the communication processed data to the lightemitting device driver with existing light communication technologies,the disclosed exemplary embodiments embed the communication processeddata on the eye insensitive part of the frequency coefficients in thespatial domain of the original image or the video when lightcommunication is performed. Therefore, the disclosed exemplaryembodiments may perform image or video display and light communicationsimultaneously.

It will be apparent to those skilled in the art that variousmodifications and variations can be made to the disclosed embodiments.It is intended that the specification and examples be considered asexemplary only, with a true scope of the disclosure being indicated bythe following claims and their equivalents.

What is claimed is:
 1. An apparatus for data embedding in lightcommunication, adapted to a transmitter, said apparatus comprising: adata embedding unit arranged between a transmitter-side communicationdata processing unit and a light emitting device driver in thetransmitter; wherein said data embedding unit embeds a communicationprocessed data at a spatial domain of an original image according to amodulation scheme, and gets multiple RGB values for a communication dataembedded image.
 2. The apparatus as claimed in claim 1, wherein saidapparatus further includes a preprocessor that performs a pre-processingprocedure before embedding the communication processed data according toan external indicator.
 3. The apparatus as claimed in claim 1, whereinthe data embedding unit generates a communication modulated image byembedding the communication processed data at one or more frequencycoefficients of a frequency domain transform of the original image. 4.The apparatus as claimed in claim 2, wherein said preprocessor transfersa plurality of pixel arrays of the original image from a first spatialdomain to a second spatial domain, and performing a discrete cosinetransform on the plurality of pixel arrays in the second spatial domain.5. The apparatus as claimed in claim 2, wherein said preprocessortransfers the plurality of pixel arrays of the original image from aspatial RGB domain to a spatial YUV or YCbCr domain and performs adiscrete cosine transform on the plurality of pixel arrays in thespatial YUV or YCbCr domain.
 6. The apparatus as claimed in claim 3,wherein the data embedding unit generates the communication dataembedded image by embedding the communication modulated image in theoriginal image in a frequency domain, performing the frequency domaintransform, and transferring a plurality of luminance and chrominancedata to the multiple RGB values.
 7. The apparatus as claimed in claim 1,wherein the data embedding unit embeds the communication processed databy inserting one or more frames modulated by one or more imageintensities inter a plurality of original frames over the originalimage.
 8. The apparatus as claimed in claim 7, wherein the one or moreframes inserted are modulated by one or more intensities of one or morepartial images in different regions over the original image.
 9. Theapparatus as claimed in claim 3, wherein the data embedding unitperforms a domain transform from a frequency domain to a spatial RGBdomain for the communication modulated image, and transfers a pluralityof resulted luminance and chrominance data to the multiple RGB values,and generates the communication data embedded image by embedding themultiple RGB values in the original image.
 10. The apparatus as claimedin claim 3, wherein the data embedding unit performs a domain transformfrom a frequency domain to the spatial domain for the communicationmodulated image, and generates the communication data embedded image byembedding a transformed result with the original image and transferringa plurality of resulted luminance and chrominance data of a dataembedded image to the multiple RGB values.
 11. A method for dataembedding in light communication, adapted to a transmitter, said methodcomprising: arranging a data embedding unit between a transmitter-sidecommunication data processing unit and a light emitting device driver inthe transmitter; embedding, by using the data embedding unit, acommunication processed data at a spatial domain according to amodulation scheme to generate a communication modulated image; andperforming an inverse transform and a domain transform that transformsthe communication modulated image from a frequency domain into thespatial domain, and getting multiple RGB values for a communication dataembedded image.
 12. The method as claimed in claim 11, wherein saidmethod further includes a pre-processing procedure before embedding thecommunication processed data according to an external indicator.
 13. Themethod as claimed in claim 12, wherein said pre-processing procedurefurther includes: transferring a plurality of pixel arrays of anoriginal image from a first spatial domain to a second spatial domain.14. The method as claimed in claim 12 wherein said external indicator isgenerated by using a software programming register or a hardwareelement.
 15. The method as claimed in claim 11, wherein said methodfurther includes: producing the communication modulated image byembedding the communication processed data at one or more frequencycoefficients of a frequency domain transform.
 16. The method as claimedin claim 15, wherein said method embeds the communication processed dataat one or more frequency coefficients of any combination of a Y, a U anda V parts in a spatial YUV domain of one or more discrete cosinetransform (DCT) blocks according to a modulation coding scheme.
 17. Themethod as claimed in claim 16, wherein the communication processed datais embedded with one of a plurality of schemes including embedding aspread spectrum pattern, embedding a QPSK pattern, using a BPSKmodulation scheme in an interleaved fashion, and using a mixed BPSK andQPSK modulation scheme.
 18. The method as claimed in claim 11, whereinsaid method embeds the communication processed data by inserting one ormore frames modulated by one or more image intensities inter a pluralityof original frames over an original image.
 19. The method as claimed inclaim 18, wherein the one or more frames inserted are modulated by oneor more intensities of one or more partial images in different regionsover the original image.
 20. The method as claimed in claim 15, whereinsaid method further includes: producing, by the data embedding unit, thecommunication data embedded image by embedding the communicationmodulated image in the frequency domain, performing the frequency domaintransform, and transferring a plurality of luminance and chrominancedata to the multiple RGB values.
 21. The method as claimed in claim 15,wherein said method further includes: performing, by the data embeddingunit, a domain transform from the frequency domain to a spatial RGBdomain for the communication modulated image, transferring a pluralityof resulted luminance and chrominance data to the multiple RGB values,and producing the communication data embedded image by embedding themultiple RGB values in an original image.
 22. The method as claimed inclaim 15, wherein said method further includes: performing, by the dataembedding unit, a domain transform from the frequency domain to thespatial domain for the communication modulated image, and producing thecommunication data embedded image by embedding a transformed result withan original image and transferring a plurality of resulted luminance andchrominance data of a data embedded image to the multiple RGB values.23. A receiving apparatus for data embedding in light communication,comprising: an image sensing device that detects a transmitter-sidecommunication data embedded image transmitted by a light emittingdevice, and generates a receiver-side communication data embedded image;an image warping unit that compensates a deformation of thereceiver-side communication data embedded image by performing an imagewarping processing on the transmitter-side communication data embeddedimage, and outputs a warped communication data embedded image; and adata extraction unit that extracts a communication processed data fromthe warped communication data embedded image.
 24. The receivingapparatus as claimed in claim 23, wherein the image warping processingperformed by the image warping unit include a coordinate conversion on aplurality of pixel data in an area of the transmitter-side communicationdata embedded image.
 25. The receiving apparatus as claimed in claim 23,wherein said data extraction unit transfers a plurality of pixel dataover the warped communication data embedded image from a first spatialrepresentation to a second spatial representation, divides the warpedcommunication data embedded image in the second spatial representationinto a plurality of blocks in a spatial domain, performs a frequencydomain transform on each of the plurality of blocks, and extracts one ormore bits that are embedded in one or more frequency coefficients todecide the communication processed data.
 26. The receiving apparatus asclaimed in claim 25, wherein said data extraction unit performs amajority voting, demapping or pattern matching on the one or more bitsextracted of the one or more frequency coefficients.
 27. A receivingmethod for data embedding in light communication, comprising:configuring an image warping unit and a data extraction unit between animage sensing device and a receiver-side communication data processingunit; detecting a transmitter-side communication data embedded imagetransmitted by a light emitting device and generating a receiver-sidecommunication data embedded image, by using the image sensing device;performing an image warping processing on the transmitter-sidecommunication data embedded image and outputting a warped communicationdata embedded image, by using the image warping unit; and extracting acommunication processed data from the warped communication data embeddedimage, by using the data extraction unit.
 28. The receiving method asclaimed in claim 27, wherein the image warping processing furtherincludes: performing a coordinate conversion on a plurality of pixeldata in an area of the transmitter-side communication data embeddedimage; and recovering the plurality of pixel data of thetransmitter-side communication data embedded image by a plurality ofpixel data of the receiver-side communication data embedded image. 29.The receiving method as claimed in claim 27, wherein extracting thecommunication processed data further includes: transferring a pluralityof data points over the warped communication data embedded image from afirst spatial representation to a second spatial representation;dividing the warped communication data embedded image in the secondspatial representation into a plurality of blocks in a spatial domain,and performing a frequency domain transform on each of the plurality ofblocks; and extracting one or more bits embedded in one or morefrequency coefficients to decide the communication processed data. 30.The receiving method as claimed in claim 29, wherein a majority voting,a demapping or a pattern matching is performed on the one or more bitsembedded in one or more frequency coefficients.
 31. A lightcommunication system, comprising: a transmitter that embeds acommunication processed data at one or more frequency coefficients of afrequency domain transform of an original image, performs an inversetransform and a domain transform on a communication modulated image, andforms a communication data embedded image by embedding multiple RGBvalues in the original image; and a receiving apparatus that detects thecommunication data embedded image transmitted by a light emitting deviceand generate a receiver-side communication data embedded image, andperforms an image warping processing to recover the communication dataembedded image by the receiver-side communication data embedded image,and extracts the communication processed data from a warpedcommunication data embedded image.
 32. The system as claimed in claim31, wherein the transmitter further include a light emitting devicedriver to drive the light emitting device to transmit the communicationdata embedded image.
 33. The system as claimed in claim 31, wherein saidtransmitter further includes a preprocessor that performs apre-processing procedure before embedding the communication processeddata.
 34. The system as claimed in claim 33, wherein the transmitterperforms the pre-processing procedure according to an external indicatorgenerated by a software programming register or a hardware element. 35.The system as claimed in claim 33, wherein said preprocessor transfers aplurality of pixel arrays of the original image from a first spatialdomain to a second spatial domain, and performing a discrete cosinetransform on the plurality of pixel arrays in the second spatial domain.36. The system as claimed in claim 31, wherein said the communicationdata embedded image is formed by embedding the communication modulatedimage in a spatial RGB domain with the original image in the spatial RGBdomain.
 37. The system as claimed in claim 31, wherein the receivingapparatus performs a coordinate conversion on a plurality of pixel datain an area of the communication data embedded image, and recovers theplurality of pixel data of the communication data embedded image by aplurality of pixel data of the receiver-side communication data embeddedimage in the image warping processing.
 38. The system as claimed inclaim 31, wherein said receiving apparatus transfers a plurality of datapoints over the warped communication data embedded image from a firstspatial representation to a second spatial representation, and dividesthe warped communication data embedded image in the second spatialrepresentation into a plurality of blocks in a spatial domain.
 39. Thesystem as claimed in claim 38, wherein the receiving apparatus performsa frequency domain transform on each of the plurality of blocks, andextracts one or more bits embedded in one or more frequency coefficientsto decide the communication processed data.
 40. A light communicationmethod, comprising: in a transmitter, embedding a communicationprocessed data at one or more frequency coefficients of a frequencydomain transform of an original image, performing an inverse transformand a domain transform on a communication modulated image, and forming acommunication data embedded image by embedding multiple RGB values inthe original image; and in a receiving apparatus, detecting thecommunication data embedded image transmitted by a light emitting deviceand generating a receiver-side communication data embedded image,performing an image warping processing to recover the communication dataembedded image by the receiver-side communication data embedded image,and extracting the communication processed data from a warpedcommunication data embedded image.
 41. The method as claimed in claim40, wherein said method performs a pre-processing procedure beforeembedding the communication processed data according to an externalindicator.
 42. The method as claimed in claim 40, wherein the imagewarping processing includes a coordinate conversion on a plurality ofpixel data in an area of the communication data embedded image.
 43. Themethod as claimed in claim 40, wherein in said receiving apparatus, saidmethod further includes: transferring a plurality of pixel data over thewarped communication data embedded image from a first spatialrepresentation to a second spatial representation; dividing the warpedcommunication data embedded image in the second spatial representationinto a plurality of blocks in a spatial domain, and performing afrequency domain transform on each of the plurality of blocks; andextracting one or more bits embedded in one or more frequencycoefficients to decide the communication processed data.
 44. The methodas claimed in claim 41, wherein performing the pre-processing procedurefurther includes: transferring a plurality of pixel arrays of theoriginal image from a first spatial domain to a second spatial domain.